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ABSTRACT , 

/ Using as an exanvnle the development of the 

comprehensive school concept in The Netherlands', this paper explores, 
the strategy for educational innovation in a centralized national 
educational systejn, the role of .educational research and evaluation 
in educational innovation, the impact pf research on policy-making, 
and the strengths and weaknesses c& educational research and, 
evaluation for ' developing- pol icy . The comprehensive school concept 
involves increasing the number of years of basic education for 
students while .postponing the time when they will be tracked into 
specifically academic or vocational high schools % by approximately 3 
years. The paper analyzes the effects df changes in government on the 
availability of support for experimentation and discusses the 
failings of a strategy for development that called on 'the 
experimental schools to develop programs independently while 
Contributing to the national effort; The types and functions of , 
- research necessary for implementation and evaluation of an innovation 
are considered next, followed by descriptions of the research done on 
the comprehensive school experiment and 'off the purposes to which the 
research was put. The paper concludes with an analysis of the factors 
Effecting innovations and offers recommendations for more effective 
research, implementation ,* and policy-making strategies. (PGD) 
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f : 7i LDiL?:; op hicgza^p:!: the case of t?ts ^utch no;^RKnF::3i7E school:; 

:::TLOj;cTir:: 
- / 

Tor -any years politicians and educational res etchers anl nraotitionerc 
have harbcrei'the belief that systematic research and evaluation stuiioo 
v.ill ^ro^iie a sound, scientific basis upon which to make iecisions 1 abo ,i 
tne usefulness of propose! educational pro-rrarrs. This belief be{:an t-o receive 
I ^ r '"n"iicrab;e support in the" USA in the nd-IJcCs. 

1 This flowered into an enormous evaluation movement and. subse-niontly into the 

C" r r.as is upon policy research in many programs in education administration 
rtT.A research methods. The initial period of emotional excitement about this 
\ v.eidir.r of research to adrrn nistrat ive and political decision makinr be ran to 

fait as the realities of life in, the schoolboardroom and the legislative 
hails became kno$.*n tot evaluators and policy researchers, 
'It has become apparent that the. role of research" and evaluation has to be 
> redefined - that research, no matter how well conducted of clearly explained, 

often rets list or ted, abused, or even irnored when fundamental, and often 
far-reaching, decisions are made*. The policy research and evaluation 
communities are now enraged in the ^complex task' of redefining their rol^e 
I and "othods so as to maximize their (potential) impact. 

In the Netherlands two far reaching education innovations started m the 
1970s: the innovations of the primary school and the comprehensive school. 
We will deal with the corrn^rehensive school (Middenschooi ) only. 
^ Trie innovational process and the role of educational research, in it was ' 

partly based on the experience of others (Sweden, USA) and partly on a 
different definition of the role of research and "evaluation in the Dutch 
educational. innovation. Now that some time has passed, some questions can , 
^ be stated when comparing the situation in the USA and the Netherlands: 

- the stratery for educational innovation in a more centralised system (in 
the Netherlands) s ^ 

- the role of educational research/evaluation in educational innovation 
^ - bh'c impact of research in policy decision-making 

- the strength and weakness of educational research/evaluation. 

This paper will explore the questions named^above in the context of Hhc 
development of comprehensive schools and the role of research in' it. 
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che'::*e of. the p^ser iz as follow-: * 

a brie:* ieccription of the development of the Dutch comprehensive school 

% r *e eiucatiofcal research and evaluation in the evaluation process 

a proposal for the improvement of the irr.pacH-of educational research. 
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A rPir? DL3^RIPTI0:: OP THE DE-/ELOPIC?:T CF Thg DUTCH CfATi?:i""?;nTY" tT^oni 
1.\ A SL'KiABY 0? THE POLITICAL DECISIONS *• 

The e ducat iona^ -secondary school syster in* the Netherlands, is" very sclented. 
There are diffprent types or' schools, x'or general education and 'for vocational' 



:i^aticr., eatery v for different altitudes' of the 'students, 
.r.j ! arrocta't, the secondary education act of^^Co, rade sore uyproveijents, 
th v.e scr.oolsysterr, for example lowering the thresholds betv;een the different 
s ' ;.o c -types • -?ut trie syster as a whole regains thp sarne. 

In The late *560s, early 19 7 0s, there was a ^Teax deaf of ^riticicn on the 

r: c c cr.u ary :; o r.oo 1 f r or t h e rart of e J \ n a* i c nal tir. o c r i s t s an i o rj r : : on a 1 

research ani the labour unions. This criticism- was basal on research Reports 

on tr.e participation of different socaal classes in education, ddeas 

crnc^rnmr equal opportunity, ideas concerning: the lack of rrotivatiori in 

:v,;on:^ rue to the curricula, ideas eoncerning tne gap between what was 

learned m schools arid- what was* going, on m society. , 

Ir.c* ..oluticn to the problerrs was to be, just as in other * countries *in Europe 

( llhe 0 err an and CwedSn, where already a different secondary school system 

existed), the Gorprchensive school. When in 1974 the Netherlands got ' a central^ 

. left government coalition^ (socialists and christian-democrats ) , this idea \ 

/ found a political basis. The Minister of Education presented *to Parliament , 

a iiscussion memorandum 'Contours of future education , in the Netherlands' , 

- * * 

(l' r ~), which contained tjie main principles for the comprehensive school, 
■namely, * * ' 

- The choice of further study and career is postponed by three or four 'years. 

Nowadays, in ^he selective categorial educational system, children' generally 

have to choose at the age of 12 or 13, » which usually iias, radical consequences 

for. the rest of their lives. In the comprehensive ■ scfeo.ol „ the choice *i'S ' ■ 

postponed until as close as possible to tRe^age of 15, 

§ > > \ *, 

- The efforts of the primary school tQ offer equal, i.e. optimal, opportunities 

* lo all chilflren for development must be continued. It is axpeot,ed that a * 
contribution can be marie by reducing any disadvantages t^usQd. by home 
background and by ^thc limitations** of the traditional Tcmajt) role. 

- The contents of the education offered to 12 to 1 6 year olds must be % 
broadened. This can be achieved by increasing the nufr?b ft er f pf subjects and 
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oifering opportunities for a^rr-ore balanced and less narrow aevelonrnent of 

I 

ail the skills the children have. These explicitly include the creative or 

s 

artistic, organisational and social skills. 

- .Suitable teaching and learning situations must be offered for individual 
' kveloprcnt and the stimulation c> social awareness. 

""re *,har. h^t,nerto, education can help people to "^top rQgardin/> society and. 
v - ir* place m it as a : v regono conclusion, and to start realizing that they 
<*an 10 sonotning about it. A ' ' t 

tncr.c was a lot of disagreement about the idea's* behind the comprehensive 
e: <ca:^cji and. a comprehensive school sy stern a:> well, it was impossible to - 
ir.:ro.uce a corrprehensiye education act at th£ sameHime. But in the 
-• r "Vcrlar.is it is possible to carry out experiments in eiucation on the basis 
\-i a special act (the so called experimentation .act), ^enabling schools to try 
t\ ' caacnal ncas m practice. In this case it means that comprehensive 
eiucation is not , provided Y or by act -but i-s going to be the result ef an 
experimentation process. On the basis of the experinentati.cn act ^cnools can 
be involved in an educational change process, relatively outside the ordinary 
school systenr. In 19&5> after a period of ten years of experimenting w^ith 
comprehensive education, there is to be an evaluation partly baspd o» results 
oi educational researcn and, evaluation. Parliament will then have to decide 
on- the legislative introduction of the comprehensive school. 
In 1975 Parliament agreed On the development of comprehensive education 
within the boundaries of the experimentation act, after the Minister of 
Education had assured the opposing conservative party that ^Ln the experiment 
attention would be paid not only to the question of how ta organize the 
comprehensive school, but also to the question of social desirability and / 
educational realization (Leune, 198l t p. 381). In f^ct not much time is 
cpeni^in Parliament on discussing the main principles Of the comprehensive 
school and the goals of the 'experiment. No criteria were developed or 
indicated on whi^^Parli^ment yill be able to decide in 1985- About one year 
.after the start^^Jthc experiment in 19?6, 'there was an * change in the* ' ' 
government coalition from ben tral-lcf t to central -jjight t (christian-democrats 
and conservatives). Thc^conservat ive part// was against the development of a 
f'onjpfehcnc Lvc> school replacing the whole secondary educational^ sys t$m and 
was in favour of a more differentiated segmented school system for secondary 
education. The conservative Minister of Educatio^d^ not actually stop the 



the^cxporinontal process but neither did he give it new impulses. He did 

withdraw 'some funds whip}^ supported the innovation 'process (less money for 

research ani development,) • Furthermore he wrote -his *own discussionpaper for 

iuture secondary education (1^80) (which conflicted with the idea of 

I corpreher.c*ve education. He proposed \o postpone the choice of further study 

\^ni career by two years (in different types of schooling) insteaa. of three 

fo'ir years fcr the duration of the comprehensive school (in orje Type of 

spooling) and Tie pleaded for more Emphasis on the cognitive subject matters 
• ... * * 

wf'nin school .education. Those in favour of the comprehensive school did not 
a/Tec with *hc proposal; therefore the discussion "became polarized. But iti 
tnc r can*: ire the development of the comprehensive school did not make much 
rrc/rc:;s. In 19'" 1 there w^s again a change of ^overnmeni irffco a central-left 
coalition, witn the same Kinisxer of Education as four yeers earlier. Because 
of "ho instability of this government (inhere is only a small, uncertain 
cajority) t:ie Minister 'of Education wanted to secure the principles of the 
comprehensive^ school as st*on as possible'* He published a new" proposal m last 
January, again a discussion memofaMum (Verdcr na.de basisschool, 1932). 
After one year oV discussion a" preparatory act will be presented in 1983* 
Then a period for experimentation fix a greater number of schools will be 
available, preparing for a definite act. To make the proposal more acceptable 
for the opposition in Parliament the heavy accent on individual enhancement 
aid social awareness has been left out; even the name was changed from 
comprehensive school into continued basic education for twelve to fifteen 
year olds. , „ 

As -you 'can see the .political scene has changed several times and with it the 

t 

political decision making on innovations toi "the secondary school systenu 
This influenced the function and role of research in the innovation process. 
Another boundary of educational research is set by the strategy used-. ^We will 
go into that in more detail in the next paragraph. ^ y^— > 

1.2. CHARACTERISTICS OP THE STRATEGY ' . 

> - ; 

The comprehensive -adhool experiment 'started in. 1975- The educational ^strategy 

Lhat was developed taking into consideration the disappointing results of the 

so-called R.D.D. strategy in educational innovation in tie- past (Lagerwey, 

/ v 
1981). This time a connection between development in schools and what was 



'join^ on at a national level (Ministry) Would not be neglected. The schools 
should be the central focusing point of the wliule cfiange process. The Minister 

"of Education appointed an Innovatory Committee for, the Comprehensive School 
Experiment (ICM), which had* to advise him gm the experimental process, the 
stratcgy'of chafge, the support for the experiment, and so on. Whfcn the 
exi. oriental process smarted, only two schools where! fully involved, five 

\;ciiools were partly involved. During the schoodyear 19^1-19^2 eigkt schools 

: v «Ily participate in the experimental process. In these schools the main 

principles of the comprehensive school mentioned above were supposed ^> get 

, m a concrete f 02 m by curriculum development and organization* of the school and 

learning situation. The change processes in the schools were supposed to be 

supported by professional development, guidance, oounseling, educational 

research, and £0 on. Purthermore^a lot of ' facilities were available the 

schools Vnvolved in the experiment, ^he comprehensive school experiment not* 
^ 

only aimed* at an educational innovation in separate schools, .but a 

Comprehensive school on a national level should oe develoj^d. The results, of 

the experimentation process such as curricula and types of organisation, and 

so on should be used on a national level, especially After passing, a 

comprehensive school act in Parliament. That means that the schools had to 

contribute to the establishment of comprehensive education on a national 

level, by departing in the experimentation process f rom^the same principles 

by reporting on their experience's in their schools, by their products from 

the developmental process, which can be implemented in other schools and so 
* * * 

on. 

But it did not seccee&'f* the experiment was isolated in the secondary school 
system and contribution to the r change on the national level remained small. 
We give a shori* description of the present* situation (Creemers en de Vries, 
1981). ' ' 

- The developmental construction of comprehensive education calls for a lot 
of energy and, Money* Each school has .to make *its own curriculum program, and 
also provide for iifs own development process, *>and so on. • * * 
That energy, that money is t?ght. Schools beneMt too little from each * 
other 1 s experience, each school has its own development. * , 

- The participants in the experiment schools, parents and children have 
-\ too little certainty about the results of the developmental construction 

V*s* within-', the school. It depends? too much qji incidents without syatematization /n. 



of experience and increase of' educational know-how. ■ * ^ x 

- This leads to casuistry. 'The Experiences and results 'are herd to generalize 
*or to systematize. It is impossible to -draw conclusions concerning all the 
school*, and from 'individual schools to the national le F vel and so on. Every 
tire you have jto look at an individual school, ;at the situation, in that . 
unique school; -you do not know what • to take of it, what is worthwhile. * * 
I <Phi3 means that,, in the future other schools will not "be able to take 

♦ advantage of the results, products and expediences of the scho6ls involved 

m tne experiment. /They are uncertain, too, about the results, about what 

will happen in their schools wheh they start the experiment. ^ 

J 

- In our opinion there is progress in the_ schools "but it was hard work, it 
took a, lot of -time, money and dedication o^feople. Thera was no or little ' 

, progress on* a national level, because it is uncertain whether or not the 
developments in different schools, (only a few), willjgo into ths* direction - , 
of the construction of comprehensive education. Rusher mo re it is very 
uncertain v/hether other schools, t which don't have the same conditiohs, v/hich 
io no« have extra money and expertise can turn into the same comprehensive schodl 
as these schools. Therefore the innovation process canno't "be "implemented in 
otner schools and only some of its products can "be implemented. In this 
respect there is one more major difficulty, namely that the schools involved 
m the experiment at this moment are not representative of the Dutch educational 
secondary school system. 'This disappointing situation is ^caused "by different 
factors we shall sum up in the following: 

- The vagueness of the main principles. ^ * 

The principles, as we have "been formulated in par. 1.1. were not operationalized. 
Thia was partly due to the fact tfrat there was no agreement "between the different 
interest groups. Innovatory Committee was given *the task to formulate t 

directions for the schools, how to giye the main principles a concrete for%* 
These directions were also very vague. ,There has never "been a discussion 
about the means "by which the principles could "he attained, for example: 
what kind of curriculum should- .be developed, what kind of schoolorganization 
can provide a curriculum and which educational process is more democratic and 
emancipatory, what kind bf differentiation is ^uitabele for comprehensive 
schools. -\ r 

Consequently, the schools starting off on the 'same principles, went into 
different directions (ELM, 1982). \ m ' 



- Tho innovation history of the school. 
* The schools were. given the opportunity to apply for participation in the 
experiment (Oreemerc, red*, 1977'). The innovation Committee made a selection 
of the schools using criteria as represehtativenes (different denominations,, 
types and counties). As a result of this the schools involved in the experiment 
arc lifferen.t m their pedagogical ideas, their opinions about comprehensive 
education, curricula and organization of the school/ 

Vzzt of these differenced are based on educational experiences "prior to tne 
eomnrenensive school, experiment . For almost e^ery school the experiment 
^ons-tituted a possibility to get more facilities (funds, support) to continue 
tr.oir own development. 



-.The support on the national level. 

"The strategy was intended to enlarge professional skills in schools. 'Besides, 
there would he a support system for curriculum dovelepment and guidance of 
the c 1 a^ige processes within the school. But at, the start of the experiment 
tho supporting system was not capable of full filing its task. 
In some cases, as* m the 7 case of curriculum development, there was no support 
at all. Gradually, .the support system was organized but remained relatively 
outside of what was going on in the schools. * 
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T Tne dominating position -of the schools. ' > > 

The participation or the schools" was rrecessary to give form j&o the idea 

■and principles of comprehensive education.' As the principles and the guidelines 

for the develomental process were very vague and the support for the experiment 

came- too late and remained fragmentary, the schools went on with their own 

developments and got a central % pa«rt in the experiment $ as promised by th/e 

Innovation Committee. During the experimeht""t"Ke~ sclop-ls stressed time and 

a^ain that they -alone must give* the comprehenaivei school its concrete fttrm; 

As there was no alternative there was no clear opposition against this^ poirft 

, ► 

of 4/iew. 

• ■ \ 
- Political and social support for the comprehensive scfrool. 

As mentioned before the political scene changed v several times and consequently 
the impetus for the comprehensive school. Connected with the diminishing 
political preference the 'basis for comprehensive education in society has 
become smaller since 1975 a*xl many people, especially in teacher unions of 
general secondary and grammar school edupation, were strongly against it and 

• \ :■ - 12 . ■ 



staged that way. The weekly of the teacher union * (^.O.L. )* involves a lot, of 
articles, statement s and one sided views and summaries *t>f education^ research 
(Tromp, 1981 )♦ This led to a5j isolation of comprehensive education in society. 
Their arguments : .are: too^litiile attention for good students, not every pupil 
is the same, ^impossible *to develop, impossible to implement "by teachers. { 

2. ^blJCATIONAL RESEARCH AND EVALUATION IN THE INNOVATION PROCESS * ' * 

: 1 . 

2.1. FUNCTION 1 OP RESEARCH 

Dhcre are different models, for policy making (pjire rationality model, 
decision 'making 1 as problem solving, splitting up the total ' decision task, 
Incremental ism: ocneerens, 19?0"j rnost of them are variant of the pure 



rationality model. There should he a lot of information, immediately or t 

■ later on,, pn which political decisions cak be "based. One function of research . % " ° 
* x \ • ' . * t - 

"ft* to pro*A.de the public policy maker with information on which decisions \ 

^can oe "ba^H^^ 'Furthermore "because the process of giving the comprehensive 

, «\, . " « 

school its, concrete form, started off from scratch, information is* needed 
# <Y Xo e©lve probl^^^^/ithin the developmental process (formative evaluation). 
^I^^iis means i;Rat T ^esearch has different functions in policy-making such as: 
' irtftifttive^testrximentalism and legitimisation, . * % 

Ta(^feals of research in- performing these functions are: description of the 
situation, explanation of the phenomena and evaluation of what is going on. 
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Oi/ top of" these goals ^pt ion-research is directed to^contribute to the 
innovative process (Xeune, 1981). 

Extfept for this policy-oriented research at the £tart of the innovative w 

forocess, ,two other types of research-, are distinguished: ' - 

1 • ^ 



.1, School oriented research (practitioner oriented research; should provide ^ 
individual schools with" information that they can use within their experiment 
tal process: development of programs,, organization of vthe^class, the school, . * J k • , 
teacher-pupil interaction, guidance of pupils and, SO. on; t , - . 

2. topic-oriented research chpuld^ive information tVftoth schools and * . 

more central level- (administration, Innovatory Committee) about the - 



specific ^opics, for example: developmental process, internal organization^. " 

* - • * . * * J » * • i- t ^ ^ ^^^^ 

forms Of differentiation and so on (Mens, Van den Berg, 19 j. 
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With respect to the decision of Parliament at the start of the experiment, 
research should "be evaluative to provide information for the 'final decision 
' on cornprenensive education at a national level, ^ « 

Within evaluation there are several possibilities to make comparisons 
(Raschert, 1975). ^ ' * 

a. Comparison of the actual situation within the school with its own 
innovative his'tory; . ■ 

b, comparison of different experimental schools; 

Cm comparison of experimental schools with ordinary schools; 

di comparison of experimental schools with the- Drincirsles of the comprehensive 
eaucation. 

Ax first research x was oriented to the first two levels of comparison. Later 
, or. in 4:he experimental process the two other levels of comparison would "be 
mtroiuced, especially with respect tojthe implementation of the comprehensive 
school in the ordinary school system. 

Looking at thege levels of comparison and^the different orientations of 
research, 03c can imagine that this leads to a huge program for research 
within an innovation process. 

Research should provide a description of what is going on (input, flow and 
output of comprehensive education;. Research 'should "be evaluative, ^should 
contribute^ to- the developmental process and, last but not least, topic 
oriented ^research should contribute to educational' theory about comprehensive 
education, differentiation, school organization and so on. 
The idea behind all this was that even in the development of comprehensive 
educati or], 'based on a normative orientation (ideal of equal opportunities') 
the change should be made on more or less rational basis within schools and 
on a more central level (innovatory Committedikadministration) and finally 
in more political decisions'in parliament. 

The decision on the research program and research projects within the 
experimental process was finally made by"the Minister. of Education, Following 
orgrmogram shows "the relationship between the^ various units involved' in 
deciVuon making procecces on research (Scheereno,. 1981 )• 
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Research t 
Institutes 



"Coordination Unit 11 - institute, foundation or committee that coordinates 
the* activities of schools participating in thq experiment, ^ 



J.C. 



- Innovatory Committee , ' ^ 

P.A.C. - Program Advisory, Ctfmmit.tee/( these committees develop general 

research programs )• , Y 

3.V.O. - Foundation of Educational Research 



The Program Advisory* Committee is set up by the^JKinister of Education, and 
reports directly to him. It is composed of independent experts, field 
rdpresentatives and observers from* the* Innovatory Committee, SVO and the 
Department of Education. Its task is to develop a general research program 
and to advise on the "policy-relevance" of specific research proposals 
developed within the general research program. * 

Within the context of research on innpvatory programs, SV/f acts as "mandated 
supervisor" to the :;ngtitute& -that^ca3^~-out~ th e act ual -research-projects-* — 
CVO also exercices process control over the ongoing research projects and' 
advises the Minister on the allocation of research funds, for which pur- 
pose it employs the service of externa], experts. 

The Inncrvatory Committee advises the Minister of Education both on the general 
research programs developed by the PAC and- on research proposals. 
In each sector (here: comprehensive school) there atfe different ki^ids o£ units 
v/hich coordinate the activities 'of experimental schools.- 
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In the case of the . comprehensive school this is a fairly loosely structured 
committee, consisting of representatives from schools, the Department of 
education and the Innovatory Committee. ~ 
For the experimental schools taking part in the comprehensive school 
experiment, bcth budget and program control are exercised by the Minister 
of Education^ Tie work of the various research institutes, involved in the 
policy-oriented research m cfuestictfi, is supervised by SVO; 3VD also exerciser 
"r.anclaied" budget control. 



The omorandurn "Contours of future education in the 'Netherlands" on wnich 
the development of the comprehensive school system is based suggests that there 
is a lot of lata and information about comprehensive education. In our opinion 
that '4s*not quite true, but even if there is a lot of evidence, we did not , 
gain advantage from .the inforrr^rtfion that^was available on comprehensive 
education. Not in political decision making, not in the developmental process, 
not in the educational support system within the innovatory process and so on. 
Perhaps this was the first violation of the pure rationality model in hot 
experimental process of the comprehensive school, A lot *of violations followed. 
The experimental ^process started when there was no research at all. Aft,er a 
"period of time our Research Institute in the Northern part of the Netherlands 
was 2,sked to make a description of the starting situation m the schools in- ■ 
volved in the experiment. At that time no -program advisory committee^ existed, 
which ''could criticize a research program and research proposals. We prepared 
the* research project and took into consideration the main principles of com- 
prehensive education and the guidelines for the developmental process f and 
made a- -proposal that in our opinion could serve the central level and the 
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Vchoois^and which was to provide not only descriptive information but also i 
form k "basis for „e valuation and decision making later on. Because there were 
no' clearly defined research questions, we started with an almost endless 
Jlclifioration with schools-. Innovatory Committee and* Department of Education 
about variables, ^topics and instruments. 

But we have had a> lot of quarrels with'several schools about the research 
questions, the research process and finally about the research reports. The 
organizations on a central level denied that the information was useful for 
tho^Lr^ purposes and' so did the schools. 
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Imrcliate results of these quarrels were, firstly, that a clear distinction 
in rocearch^prograjrjninc "between policy oriented research and school practitioner 
oriented research was made and that within the policy oriented research 
priority was given to a rjescriptron of the innovative situation within ^ 
scr.oo.lsi This policy oriented research should give more information aboujt 
the 'state of affairs, „ ^ 

In oiir opinion a lot of organizations, for example the Innovatory Committee 
ana some -schools were afraid' of the evaluative information of that first 
research project. (Some conclusions in that report (197?): theVe is a lack 
of professionalism within schools, lack of support, only a special group of 
students goes tot , comprehensive schools, the developmental process withn the 
schools is "based on tneir own innovation history and ix is doubtful wheeler 
# tr.at process will lead to the comprehensive education (Creemers, a. 6., 1 9 7 7 ) • 
Due xo lack of money only two schools oriented researchers could "be retained 
who carried out research in a few experimental schools* Though their work 
should contribute to all experimental schools it is not always so. Whenj the * 
structure) wa& set up for the development of the research program and tne 
research proposals it became obvious that a lot of interest groups and! orga- 
nisations nod- influence on the formulation of the program and research .questions. 
As a result of this there is a lot of negotiation between the different orga- 
nisations a^out research proprvsals. Different organizations involved in the 
innovative process had different options in res*beot to research. 
The Ministry of Education, at least the administratieve part of it, was 
only interested in the description of the situation. For the Innovatory 
Committee the main task of research was description and legitimation of what 
the Innovatory Committee had done, especially in the period of the central 
right government. It was obvious that what they did not ^want was evaluative 
research, perhaps- because they were afraid of the results. 
The Foundation of Educational Research, which mandated research, requested 
excellent research designs, quantitive methods for evaluation (which others- 
did not wantJJ if possible a contribution to educational theory; in fact . 
almost the opposite of what the Innovatory Committee and the Minister of 
F/lucation wanted. * 

The schools in the innovatory process wanted support for their own experimen- 
tal process* In their opinion they had to develop the comprehensive education 
and in their opinion " they were the most vulnerable part in the experimental 
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Ag can "be observed most of the research projects can be characterise 1 as 
descriptive studies fore shad owing - when interpreted in, a ki$d way - t a 
general evaluation plar>. 

Perhaps now a new situation arises in the presentation of the memorandum 

of the continued basic education • When we look at the* past we have to cor.clud 

that: 

J 

- After a lot of plans and ideas about research and evaluation in the* 
innovatory process only a little bit of research is carried out, mostly 
icscriptive studies and no'T orrative evaluation. A lot of research is only 
t ^cussed on the entering "behaviour of 'pupils and the opinions of their 
r^r^.ts about comprehensive education. We have little information about 
::,o toriprchensive education itself during the three years following the/" 
pupils entrance in the school. In our opinion the research is situated in 
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— Ir. cur cn/ii^n the quality of recearcr. is poorer than should have been. 
There was no systematic research, no standardized tests, no standardized 
intelligence tests and aptitude scales because schools and others were 
opposed to these with the argument, that there should be> different education 
with different goals s,nd therefore the existing tests did^^t fit. 

There were too little direct observations; at ihat time the teachers formed 
the main source of data. , ' 

As Scneerens mentioned tnere is an ideological clash between the dominant 
normative orientation of teachers and the technical rationality associated 
m in particular research procedures (Scheerens, 1981, p. 29)* 
Finally, in our opinion the influence of schools and other parties was too 
big in the formulation of the final report when it was discussed with them 
before publication. V . 

- The usefulness of research for different audiences was rather small. 
That is what we have concluded, with utmost reserve. 

Scheerens has carried out some interviews with . governmental officials^ 
teachers and members of the, Innovatory Committee. Asked about the usefulness 
of the research projects most of the interviewees responded negatively. 
This is striking, as the research design and what was going on actually in 
the research projects was a result of the negotiating process with all' these 
people and. yet afterwards they disagreed with the results^ They had 7 
different reasons arid on top of these reasons research itself had another 
reason: the quality of ihe research* . - 
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2„3; SOS REASONS FOR THE LITTLE IMPACT . ^ 

Between the lines we have mentioned above som6 reasons for the little wpact 
of research \n the process of experimentation and political decision making. 
We will sumn^rize them in the following: 

1* The reality of rational decision making leads to evaluative anxiety In 
organizations and persons involved in the experimental process, especially 
wr.cn this ir connected with a strategy which not clearly defines the goals ' " % 
ar.i reans of the innovative process and the functions of different organ i- 
nations involved in tne experimental process; 

!• ir. fact in tne innovative strategy used, there is no room for research * 
*ith ixs owr.^ qualities. Especially the place of the schools in the strategy 
leais to objections and oppositions against research; 

3. rxst of the time ti^e results of research came too late. That was not onjy 

tl e responsibility of researchers but mostly of other participants in the 

t 

innovatave process; 

» * s * 

y> r looking at almost everything involved in the innovation process, 
research has lost j.ts own objectives. Researchers have to do good -research 
using their own professional orientation, formulating good research questions, 
making good designs a^id using or developing good instruments; 
5» there is a discrepancy between centralized policy making and bottom-up '* 
change strategies* Perhaps research is a victim of an unsuccessful combination 
of a centralized and, at the same time, decentralized innovative process; 
6. expecting immediate and* useful results of research and making research 
programs in the negotiation process guided by this expectation .and* also the 
-researchers conviction that, acting accordingly, the contribution of educational 
research to educational practice leads in fact to poor quality of research that 
confirm everybody's ideas about the innovative process, its own contribution 
to it and* the no^so flattering opinions aftout i educational -research. 



3. TVPROVKKKNT of ras UP ACT OF EDffGflTIONAL research 
.CONCLUSIONS AIID SOKE RECOBCT DATIONS •. 



3.1. POLITICAL DECISION MAKING' ."AND. CHANGS STRATEGIES 



/ 



VJitn respect to education thp,*political system in the Netherlands is 
centralized^ TMs means that the government provides £oney' f or education 
and sets up standards for education, not only for public schools, "but for 
thu private schools for different denominations, too. Within the framework . 
of t.'.ese standards, whicK / contain, the educational structure, the goals that 
have to be achieved by education, hy» schools, public and private as v/ell,^ 
the schools .can work out education in their own way. Therq. are, in- our 
opinion, several 'goo'd reasons for .this centralised system: the Netherlands 
are relatively small, one avoids extreme differences between private and 
rublic schools an£, most of all, education is so important that J. t has to 
be in the continuous care of the government. Several discussion memorandums, 
for example "Contours of the future education in the Netherlands" show that 
the government takes its role and function concerning education seriously. 
As we have mentioned before there is a lot of disagreement on the value of 
"he goals and structure^ of the present educational system and the main 
principles of educational innovation. 

In essence this discussion is a political one, though a lot of different 
interest groups quarrel about these topics. Educational research plays /a 
small part in' the discussion. Different opponents make use of the research 
results for thei»own purposes-. As we have seen before, formulation of a 
research program and of research questions is quite difficult in such a 
situation. 

For the implementation of^e&ucational innovation it it not necessary to 
wait for the consensus of optegpn in the society. But it has to be clear who 
states the research question and who is interesteol'in the research results. 
Educational research, cannot be the negotiator between different interest 
groups. , , 

Even though it is nd^Jjiecessary to wait for a consensus, actual implemen- 
tation of educational innovation requires a relatively stable government, 
especially in the situation of 'the comprehensive school, where there is 
an enormous disagreement . 
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^The innovation, of primary education, the so called "basic education for 4 
to 12 year olds, is being carried out likewise by different governments. 
In literature different strategies are available for educational innovation 
and li^eir Lnation (implementation and adoption),* Well known' is the R.D.D. 

^strategy on which a lot of educational innovation in the United States and 
the Netherlands is based. When this strategy is practized rigidly,, results 
are disappointing, especially in the disseminata phase. Therefore new 
strategies emerged, such as the problem solving strategy and more bottom- 
up strategies for educational innovatidH^One more reason for this kind 
c: strategies was the ideal in the Netherlands to go back to the basis, 
the iomocratization of education in development, practice, and government 

' of tr.e school. 

In tnc innovation of primary and secondary education the Innovatory Committee 
tric^ to i?iake a combination 1 o£ R.D.I}., top-down and bottom-up strategies. 
^In tr.e case of the comprehensive school it dit not succeed. As showed earlier 
.the bottom-ftp strategy prevailed over a more R.D.D. strategy. 
As we have mentioned before w$ have serious problems with the stititegy used 
in the development of the iJutch comprehensive school. All in, we suppose 
that a flexible form of R.D.D. can bemused in educational innovation in a 
situation as in the Netherlands. The intentions of that model are that better 
.use can be made of professional knowledge in each level (researph, development 
and teaching) and thit everybody contributes to the educational innovation. 
Flexible me^ns that research, development, implementation and dissimination 
are not separate, activities but activities' 'that can^be combined within the 
different activities several professional experiences are combined^ too. 
This is efficient in terms of /noney, manpower and time. 

In our opinion, 'with respect* to education, there is a discrepancy between 
the centralized political system and bottom-up strategies". A solution has to 
be found for the problem^of the discrepancy between central guidelines for 
the development of education and its operationalization in schools. On the 
other hand the bottom-up strategy is very inefficient, cosite a lot of money,, 
tends to ignore professional support and furthermore implementation in other 
schools docs not succeed* , 

Educational research and evaluation can be a victim of s discrepancy 
when it gets .demands from both or even more sides, as we have mentioned 
before • The central government wants a summative evaluation but cannot 
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formulate the -guideline::, the goales, the*crit®ria this kind of evaluation 
should "be based on. On the other hand it neglects research reports that 
contain more formative evaluation of what is" going on in different schools. 
The individual school wants support for its bwji developmental procecs and 
neglects at the same time the contribution' of its own developmental process 
to the national educational 'development . In that case educational research, 
dhich doesn't want to limit itself to one particular situation, fcoes not fit. 
In centralized political syst£m§, schoolt?ased development ffLone does not fit; 
we have to build a relationship between this schoolbased change 4 and what. is. 
roinr on on a .centr.al • level. When a political cistern is more decentralized,^ 
a:: in the United States, more bottom-up strategies will be suitable. 
Research is'ramly connected with what i$ going on i n schools by means of 
formative evaluation. The question is- whether more sutfmative kinds of 
evaluation could be used to give the government more insight into what is 
happening into successes and failures. • % 

If xhl3> problem of the discrepancy between the government and the 
concretiration of innovation can be solved in a more flexible RDD-npdel, 
educational research and evaluation will be able to Contribute to a more 
central innovation of the whole educational system and developmental 
processes m individual schools as well, providing information about the 
concretization of innovative ideas on both levels separately and in* a 
relationship with each other. ^ 



3.2. FUNCTION OF THE RESEARCH 

A- have sr-en educational research with respect to the development of the 
Dutch comprehensive school plays a role of minor importance in (political) 
decision making. This is not particularly so in this innovation process, but 
a lot of publications are about its dissappointing role. These publications 
contain proposals on how to increase the role of educational "research 
(Cideoncon, Ilabcrlin, 1970). In the case of the comprehensive educational 
experiment several improvements can be made. - * 

1. In order to change the conditions we need more insight into political 
tfecision making, for example using experiences gathered during the 
comprehensive school experiment. In a careful analysis of the decision 
making processes ^the decision points, the criteria, the information needed - 
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p*on, can be found. After that the relationship between research and 
ion making can be improved. 

decision has to be mad^e whether or not research is suitable m 
particular situations. It is not necessarily the task of research to gat he* 

f 4 

the" opinions of people such as interest groups, teachers, parents, pupils, 
tr.e employers pad employees. The discussion in society on which political 
iccisions can be based is sometimes 'started by educational research but 
.mostly by journals, opinionpapers and so on. Educational research or 
opinLonresearch can sometimes contribute to this discussion by providing 

. L ...... . ■ 

more oojective information, 

3« Formative evaluation can contribute to the developmental process on 
different levels, especially in- botiom-up strategies, but also in Lop-down 
strategies. 

Research provides a'basis for rational decision making within the schools 

ani provides the government with information .on which decisions can be based, 
> 

Furthermore we get information on the conditions, processes and results in 
the experimentation process which can be used in the dissemination of the 
educational innovation. Later on in the experimentation process more 
summ*,ti4re evaluation can be introduced. But some discrepancy between' 
summative evaluation and bottom-up strategies will remain, 

4. By means of a mitigated flexible RDD-strategy rationality can improve 
political decisicfrr making; results of educational research, ^including 
summative evaluation, hav§ a better chance to contribute to it, 

5. Our experiences in the comprehensive education' experiment poiijt out that 
sometimes research is not necessary and that rthe same job can be done m a 
school level «by the principal, or school administration and on a centrkl 
level by the Ministry of Education, 

6. We haVe pointed out that research runs thq risk of playing lipservice to 
different groups ,when there is a controversy^ central level versus school 

JLevel, different groups and so on). • | 

In our ooinion educational research always, ^ind especially in such 

i 

politically controversial situations, has to 1 be critical, next to its other 
function 3 already mentioned; its criticism bpin^ related to these functions, 
if possiole. ' I # 

Critical educational research can show the advantages and disadvantages of 

1 J 

educatio|nal innovation in terms of conditions, jgrocesses and outcomes 

? 
o 



This requirec a relatively pure position of spcial science in society. In 

this case educational research has indeed an enlightning function ( Weiss, 

), debunking of ideas and opinions that are available in -society about 

education. In the case of the comprehensive school this means that 

educational research has to formulate its '(own) research questions, (own) 

criteria, (own) goals, ^based on the analysis of literature, what is going 

or\ in society and in public policy. Research ddes not have to wait for the 

whole process of goals and ideas to get its concrete form. Thi's is what 

^research usually does, consequently having to provide alibis for the points 

of view of all 'parties involved and having to find arguments that "Nfttradict 

the opinion that research has no use. When researchers do their own job 

society will, listen to their arguments. We have one more argument supporting 

that statement, based on research- in the comprehensive school. When we 

s 

carried out a second' analysis on the material that we gathered in different 

descriptive studies, we pointed out why the change strategy failed, what 

! 

went wrong t |n the developmental processes and change processes, and we put 
forward suggestions for some changes An the innovative strategy. And we 
have £0$ an audience: public policy makers, journals and professionals in 
education. 

Even when policy pursues rational, decision making very often decisions are 
not rational and then the function of research is a critical one, relatively 
independent cjf thd other ^parties involved. 
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